Allocating acceleration component functionality for supporting services

ABSTRACT

Aspects extend to methods, systems, and computer program products for allocating acceleration component functionality for supporting services. A service manager uses a finite number of acceleration components to accelerate services. Acceleration components can be allocated in a manner that balances load in a hardware acceleration plane, minimizes role switching, and adapts to demand changes. When role switching is appropriate, less extensive mechanisms (e.g., based on configuration data versus image files) can be used to switch roles to the extent possible.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.14/752,800, entitled “Allocating Accelerating Component Functionalityfor Supporting Services” filed on Jun. 26, 2015, issued as U.S. Pat. No.10,270,709, the disclosure of which is incorporated herein in itsentirety by reference.

BACKGROUND

Computer systems and related technology affect many aspects of society.Indeed, the computer system's ability to process information hastransformed the way we live and work. Computer systems now commonlyperform a host of tasks (e.g., word processing, scheduling, accounting,etc.) that prior to the advent of the computer system were performedmanually. More recently, computer systems have been coupled to oneanother and to other electronic devices to form both wired and wirelesscomputer networks over which the computer systems and other electronicdevices can transfer electronic data. Accordingly, the performance ofmany computing tasks is distributed across a number of differentcomputer systems and/or a number of different computing environments.For example, distributed applications can have components at a number ofdifferent computer systems.

BRIEF SUMMARY

Examples extend to methods, systems, and computer program products forallocating acceleration component functionality for supporting services.A service manager monitors characteristics of a plurality of hardwareacceleration components in a hardware acceleration plane. The hardwareacceleration plane provides a configurable fabric of accelerationcomponents for accelerating services.

The service manager allocates a group of interoperating accelerationcomponents, from among the plurality of hardware accelerationcomponents, to provide service acceleration for a service. The group ofinteroperating acceleration components is allocated based on themonitored characteristics in view of an allocation policy for thehardware acceleration plane. A role at each acceleration component inthe group of interoperating acceleration components are linked togetherto compose a graph providing the service acceleration. The servicemanager maintains an address for the graph so that the service canrequest hardware acceleration from the group of interoperatingacceleration components.

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

Additional features and advantages will be set forth in the descriptionwhich follows, and in part will be obvious from the description, or maybe learned by practice. The features and advantages may be realized andobtained by means of the instruments and combinations particularlypointed out in the appended claims. These and other features andadvantages will become more fully apparent from the followingdescription and appended claims, or may be learned by practice as setforth hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and otheradvantages and features can be obtained, a more particular descriptionwill be rendered by reference to specific implementations thereof whichare illustrated in the appended drawings. Understanding that thesedrawings depict only some implementations and are not therefore to beconsidered to be limiting of its scope, implementations will bedescribed and explained with additional specificity and detail throughthe use of the accompanying drawings in which:

FIG. 1 illustrates an example architecture that includes a softwareplane and a hardware acceleration plane.

FIG. 2 illustrates an example architecture, including servers, that canbe used in a data center

FIG. 3 illustrates an example server.

FIG. 4 illustrates an example server.

FIG. 5 illustrates an example service implemented using components of asoftware plane and components of a hardware acceleration plane.

FIG. 6 illustrates an example service implemented using components of asoftware plane and components of a hardware acceleration plane.

FIG. 7 illustrates an example architecture of an acceleration component.

FIG. 8 illustrates an acceleration component including separateconfigurable domains.

FIG. 9 illustrates functionality for performing data transfer between alocal host component and an associated local hardware accelerationcomponent.

FIG. 10 illustrates an example architecture of a host component.

FIG. 11 illustrates an example architecture of management functionalityfor managing services in a data center.

FIG. 12 illustrates an example architecture with additional componentsof the management functionality of FIG. 11.

FIGS. 13A-13C illustrate an example architecture for allocatingacceleration component functionality to support a service.

FIG. 14 illustrates a flow chart of an example method for allocatingacceleration component functionality to support a service.

FIGS. 15A-15D illustrate an example architecture for allocatingacceleration component functionality to support services.

DETAILED DESCRIPTION

Examples extend to methods, systems, and computer program products forallocating acceleration component functionality for supporting services.A service manager monitors characteristics of a plurality of hardwareacceleration components in a hardware acceleration plane. The hardwareacceleration plane provides a configurable fabric of accelerationcomponents for accelerating services.

The service manager allocates a group of interoperating accelerationcomponents, from among the plurality of hardware accelerationcomponents, to provide service acceleration for a service. The group ofinteroperating acceleration components is allocated based on themonitored characteristics in view of an allocation policy for thehardware acceleration plane. A role at each acceleration component inthe group of interoperating acceleration components are linked togetherto compose a graph providing the service acceleration. The servicemanager maintains an address for the graph so that the service canrequest hardware acceleration from the group of interoperatingacceleration components.

Implementations may comprise or utilize a special purpose orgeneral-purpose computer including computer hardware, such as, forexample, one or more processors and system memory, as discussed ingreater detail below. Implementations also include physical and othercomputer-readable media for carrying or storing computer-executableinstructions and/or data structures. Such computer-readable media can beany available media that can be accessed by a general purpose or specialpurpose computer system. Computer-readable media that storecomputer-executable instructions are computer storage media (devices).Computer-readable media that carry computer-executable instructions aretransmission media. Thus, by way of example, and not limitation,implementations of can comprise at least two distinctly different kindsof computer-readable media: computer storage media (devices) andtransmission media.

Computer storage media (devices) includes RAM, ROM, EEPROM, CD-ROM,solid state drives (“SSDs”) (e.g., based on RAM), Flash memory,phase-change memory (“PCM”), other types of memory, other optical diskstorage, magnetic disk storage or other magnetic storage devices, or anyother medium which can be used to store desired program code means inthe form of computer-executable instructions or data structures andwhich can be accessed by a general purpose or special purpose computer.

A “network” is defined as one or more data links that enable thetransport of electronic data between computer systems and/or modulesand/or other electronic devices. When information is transferred orprovided over a network or another communications connection (eitherhardwired, wireless, or a combination of hardwired or wireless) to acomputer, the computer properly views the connection as a transmissionmedium. Transmissions media can include a network and/or data linkswhich can be used to carry desired program code means in the form ofcomputer-executable instructions or data structures and which can beaccessed by a general purpose or special purpose computer. Combinationsof the above should also be included within the scope ofcomputer-readable media.

Further, upon reaching various computer system components, program codemeans in the form of computer-executable instructions or data structurescan be transferred automatically from transmission media to computerstorage media (devices) (or vice versa). For example,computer-executable instructions or data structures received over anetwork or data link can be buffered in RAM within a network interfacemodule (e.g., a “NIC”), and then eventually transferred to computersystem RAM and/or to less volatile computer storage media (devices) at acomputer system. Thus, it should be understood that computer storagemedia (devices) can be included in computer system components that also(or even primarily) utilize transmission media.

Computer-executable instructions comprise, for example, instructions anddata which, when executed at a processor, cause a general purposecomputer, special purpose computer, or special purpose processing deviceto perform a certain function or group of functions. The computerexecutable instructions may be, for example, binaries, intermediateformat instructions such as assembly language, or even source code.Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the described features or acts described above.Rather, the described features and acts are disclosed as example formsof implementing the claims.

Those skilled in the art will appreciate that the described aspects maybe practiced in network computing environments with many types ofcomputer system configurations, including, personal computers, desktopcomputers, laptop computers, message processors, hand-held devices,wearable devices, multi-processor systems, microprocessor-based orprogrammable consumer electronics, network PCs, minicomputers, mainframecomputers, mobile telephones, PDAs, tablets, pagers, watches, routers,switches, and the like. The described aspects may also be practiced indistributed system environments where local and remote computer systems,which are linked (either by hardwired data links, wireless data links,or by a combination of hardwired and wireless data links) through anetwork, both perform tasks. In a distributed system environment,program modules may be located in both local and remote memory storagedevices.

The described aspects can also be implemented in cloud computingenvironments. In this description and the following claims, “cloudcomputing” is defined as a model for enabling on-demand network accessto a shared pool of configurable computing resources. For example, cloudcomputing can be employed in the marketplace to offer ubiquitous andconvenient on-demand access to the shared pool of configurable computingresources. The shared pool of configurable computing resources can berapidly provisioned via virtualization and released with low managementeffort or service provider interaction, and then scaled accordingly.

A cloud computing model can be composed of various characteristics suchas, for example, on-demand self-service, broad network access, resourcepooling, rapid elasticity, measured service, and so forth. A cloudcomputing model can also expose various service models, such as, forexample, Software as a Service (“SaaS”), Platform as a Service (“PaaS”),and Infrastructure as a Service (“IaaS”). A cloud computing model canalso be deployed using different deployment models such as privatecloud, community cloud, public cloud, hybrid cloud, and so forth. Inthis description and in the claims, a “cloud computing environment” isan environment in which cloud computing is employed.

In this description and the following claims, an “accelerationcomponent” is defined as a hardware component specialized (e.g.,configured, possibly through programming) to perform a computingfunction more efficiently than software running on general-purposecentral processing unit (CPU) could perform the computing function.Acceleration components include Field Programmable Gate Arrays (FPGAs),Graphics Processing Units (GPUs), Application Specific IntegratedCircuits (ASICs), Erasable and/or Complex programmable logic devices(PLDs), Programmable Array Logic (PAL) devices, Generic Array Logic(GAL) devices, and massively parallel processor array (MPPA) devices.

In this description and in the following claims, a “role” is defined asfunctionality provided by an acceleration component to a group ofinteroperating acceleration components used to accelerate a service.Roles at each acceleration component in a group of interoperatingacceleration components can be linked together to compose a graph thatprovides the service acceleration.

In this description and in the following claims, a “graph” is defined asa group of interconnected (e.g., network connected) accelerationcomponents providing acceleration for a service wherein eachacceleration component in the group provides a portion of theacceleration functionality.

In this description an in the following claims, an “image” is defined asa file including information that can be used in configuration of anacceleration component, such as, for example, an FPGA. Informationincluded in an image file can be used to program hardware components ofan acceleration component (e.g., logic blocks and reconfigurableinterconnects of an FPGA) to implement desired functionality. Desiredfunctionality can be implemented to solve virtually any problem which iscomputable.

In this description and in the following claims, a “neighboracceleration component” is defined as an acceleration componentconfigured to exchange input and/or output with another accelerationcomponent when interconnected to the other acceleration component withina graph. Neighbor is viewed logically from the perspective of the graph.The physical proximity of an acceleration component relative to anotheracceleration component is not a determining factor in identifyingneighbor acceleration components. That is, acceleration components thatare not physically adjacent to one another (or even near to one anotheron a network or within a datacenter) can be configured to exchange datawith one another when interconnected within a graph. Accelerationcomponents interconnected within a graph can be viewed as neighboracceleration components even if data exchanged between the accelerationcomponents physically passes through other acceleration componentsoutside of the graph or through host components in transit between theacceleration components. However, acceleration components that arephysically adjacent or near to one another on a network or in adatacenter and are interconnected within a graph can also be viewed asneighbor acceleration components with respect to one another.

In general, an acceleration component can include an array ofprogrammable logic blocks and hierarchy of reconfigurable interconnectsthat allow logic blocks to be connected together in differentconfigurations to provide different functionality (i.e., differentroles). Image files can be received and loaded at an accelerationcomponent to configure programmable logic blocks and configureinterconnects to provide desired functionality (i.e., roles).

In some environments, applications (services) are provided to a clientfrom a data center. A data center includes multiple (and potentially avery large number of) software-driven general purpose computing devices.Each general purpose computing device can include one or more centralprocessing units (CPUs) that process machine-readable instructions toperform specified computing activities. The multiple general purposecomputing devices are networked to one another such that the combinedpower of the multiple general purpose computer systems (or subsetsthereof) can be used to perform more complex computing activities.

Data center providers face continuing challenges to increase processingcapabilities and efficiency within and provided by data centers.Continuing to add more and more general purpose computing devices is notfeasible due at least in part to power limitations. Computing devicespecialization is one option. For example, computing devices can bespecialized for specific scale workloads to provide some efficiencygains. However, computing device specialization is problematic for atleast two reasons. First, lack of homogeneity in a data center increasesmanagement issues and provides inconsistent platforms for applicationsto rely on. Further, data center services evolve rapidly, makingnon-programmable hardware features impractical. Thus, data centerproviders need continued improvements in performance and efficiency butcannot obtain those improvements from general purpose computing devices.

Aspects facilitate allocating acceleration component functionality forsupporting services. A service manager monitors characteristics of aplurality of acceleration components in a hardware acceleration plane.The service manager can monitor acceleration component characteristics,including one or more of: utilization and/or load within the hardwareacceleration plane, failures, performance degradation, software errors,machine liveness, service demand (e.g., surges for a particularservice), etc. Based on monitored characteristics and in accordance withan allocation policy, the service manager can allocate a group ofinteroperating acceleration components to accelerate part of a service.In general, the service manager can allocate a group of interoperatingacceleration components to facilitate one or more of: balancing loadacross the plurality of acceleration components, minimizing switchingroles at acceleration components, addressing surges in demand forparticular services, grouping based on role, grouping based on networkbandwidth requirements for a service, grouping for power efficiency,etc.

The service manager can allocate a group of interoperating accelerationcomponents in a variety of different ways. In one aspect, a dedicatedgroup of interoperating acceleration components is configured toaccelerate a corresponding service. When the service manager determinesa need for additional capacity for accelerating the service, the servicemanager allocates the corresponding dedicated group of interoperatingacceleration components to provide acceleration for the service.

In another aspect, a predetermined pool of acceleration components isavailable for accelerating services. When the service manager determinesa need for additional capacity for accelerating a service, the servicemanager allocates a group of interoperating acceleration components,from among the predetermined pool, to provide acceleration for theservice.

If a further aspect, when the service manager determines a need foradditional capacity for accelerating a service, the service managerallocates a group of interoperating acceleration components from thehardware acceleration plane to provide acceleration for the service.Allocation can be in accordance with an allocation policy or possiblyrandom.

In one aspect, one or more groups of interoperating accelerationcomponents are allocated. Each of the one or more groups providesservice acceleration for the same service. When subsequently allocatingan additional group of interoperating acceleration components, theadditional group can provide service acceleration for the same service,can provide service acceleration for a different version of the sameservice (e.g., document ranking in a different language), or can provideservice acceleration for a different service.

Acceleration components can be locally linked to host components (e.g.,CPUs), for example, in the same server. When appropriate, a hostcomponent can switch roles at locally linked acceleration components.Thus, in an additional aspect, a group of host components locallyswitches roles at a corresponding group of acceleration components toallocate a group of interoperating acceleration components foraccelerating a service.

The service manager can maintain addresses for groups of interoperatingacceleration components providing acceleration for different services.In some aspects, an acceleration component provides a role to a group ofinteroperating acceleration components that provide acceleration of aservice. Roles at each acceleration component in the group ofinteroperating acceleration components are linked together to compose agraph that provides the service acceleration. As such, the servicemanager can maintain addresses for an acceleration component at the headof each graph.

Switching roles at an acceleration component can vary from fullreconfiguration of the acceleration component (loading a new image fromDRAM or PCIe) to switching between already programmed functionality.More significant (and more time consuming) reconfiguration may beappropriate when switching between roles for accelerating differentservices, such as, for example, switching between a document rankingservice and a computer vision service. Less significant (and less timeconsuming) reconfiguration can be possible when switching between rolesfor accelerating different implementations of similar services, such as,for example, switching between a document ranking service for documentsin German and a document ranking service for documents in English. Aservice manager can minimize reconfiguration as much as possible whenallocating a group of interoperating acceleration components toaccelerate a service.

In general, a data center deployment includes a hardware accelerationplane and a software plane. The hardware acceleration plane can includea plurality of networked acceleration components (e.g., FPGAs). Thesoftware plane can include a plurality of networked software-implementedhost components (e.g., central processing units (CPUs)). A networkinfrastructure can be shared between the hardware acceleration plane andthe software plane. In some environments, software-implemented hostcomponents are locally linked to corresponding acceleration components.

FIG. 1 illustrates an example architecture 102 that includes a softwareplane 104 and a hardware acceleration plane 106. The software plane 104includes a collection of software-driven components (each denoted by thesymbol “S”) while the hardware plane includes a collection of hardwareacceleration components (each denoted by the symbol “H”). For example,each host component may correspond to a server computer that executesmachine-readable instructions using one or more central processing units(CPUs). Each CPU, in turn, may execute the instructions on one or morehardware threads. Each acceleration component can execute hardware logicfor implementing functions, such as, for example, portions of servicesoffer by a data center.

Hardware acceleration plane 106 can be constructed using a heterogeneouscollection of acceleration components, including different types ofacceleration components and/or the same type of acceleration componentswith different capabilities. For example, hardware acceleration plane106 can include FPGA devices having different respective processingcapabilities and architectures, a mixture of FPGA devices and otherdevices, and so on. Hardware acceleration plane 106 provides areconfigurable fabric of acceleration components.

A host component generally performs operations using a temporalexecution paradigm (i.e., sequentially) by using each of its CPUhardware threads to execute machine-readable instructions, one after theafter. In contrast, an acceleration component may perform operationsusing a spatial paradigm (i.e., concurrently) by using a large number ofparallel logic elements to perform computational tasks. Thus, anacceleration component can perform some operations in less time comparedto a software-driven host component. In the context of the architecture102, the “acceleration” qualifier associated with the term “accelerationcomponent” reflects its potential for accelerating the functions thatare performed by the host components.

In one example, architecture 102 corresponds to a data centerenvironment that includes a plurality of computer servers. The computerservers correspond to the host components in the software plane 104. Inanother example, architecture 102 corresponds to an enterprise system.In a further example, the architecture 102 corresponds to a user deviceor appliance which uses at least one host component that has access totwo or more acceleration components, etc. Other implementations forarchitecture 102 are also possible.

Common network infrastructure 120 couples host components in thesoftware plane 104 to other host components and couples accelerationcomponents in the hardware acceleration plane 106 to other accelerationcomponents. That is, host components can use common networkinfrastructure 120 to interact with one another and accelerationcomponents can use common network infrastructure 120 to interact withone another. Interaction among host components in the software plane 104is independent of the interaction among acceleration components in thehardware acceleration plane 106. As such, two or more accelerationcomponents may communicate in a transparent manner relative to hostcomponents in the software plane 104, outside the direction of the hostcomponents, and without the host components being “aware” of particularinteraction is even taking place in the hardware acceleration plane 106.

Architecture 102 can use any of a variety of different protocols tofacilitate communication between acceleration components over networkinfrastructure 120 and can use any of a variety of different protocolsto facilitate communication between host components over networkinfrastructure 120. For example, architecture 102 can uses Ethernetprotocol to transmit Internet Protocol (IP) packets over networkinfrastructure 120. In one implementation, each local host component ina server is given a single physical IP address. The local accelerationcomponent in the same server may adopt the same IP address. The servercan determine whether an incoming packet is destined for the local hostcomponent or destined for the local acceleration component in differentways. For example, packets that are destined for the local accelerationcomponent can be formulated as UDP packets having a specific port;host-defined packets, on the other hand, may not be formulated in thisway. In another example, packets belonging to the acceleration plane 106can be distinguished from packets belonging to the software plane 104based on the value of a status flag in each of the packets.

As such, architecture 102 can be viewed as two logical networks(software plane 104 and hardware acceleration plane 106) that share thesame physical network communication links. Packets associated with thetwo logical networks may be distinguished from each other by theirrespective traffic classes.

In another aspect, each host component in the architecture 102 iscoupled to at least one acceleration component in hardware accelerationplane 104 through a local link. For example, a host component andacceleration component can be arranged together and maintained as singleserviceable unit (e.g., a server) within architecture 102. In thisarrangement, the server can be referred to as the “local” host componentto distinguish it from other host components that are associated withother servers. Similarly, acceleration component(s) of a server can bereferred to as the “local” acceleration component(s) to distinguish themfrom other acceleration components that are associated with otherservers.

As depicted in architecture 102, host component 108 is coupled toacceleration component 110 through a local link 112 (e.g., a PeripheralComponent Interconnect Express (PCIe) link). Thus, host component 108 isa local host component form the perspective of acceleration component110 and acceleration component 110 is a local acceleration componentfrom the perspective of host component 108. The local linking of hostcomponent 108 and acceleration component 110 can form part of a server.More generally, host components in software plane 104 can be locallycoupled to acceleration components in hardware acceleration plane 106through many individual links collectively represented as alocal_(H)-to-local_(S) coupling 114.

Thus, a host component can interact directly with any locally linkedacceleration components. As such, a host component can initiatecommunication to a locally linked acceleration component to causefurther communication among multiple acceleration components. Forexample, a host component can issue a request for a service (or portionthereof) where functionality for the service (or portion thereof) iscomposed across a group of one or more acceleration components inhardware acceleration plane 106.

Thus, a host component can also interact indirectly with otheracceleration components in hardware acceleration plane 106 to which thehost component is not locally linked. For example, host component 108can indirectly communicate with acceleration component 116 viaacceleration component 110. More specifically, acceleration component110 communicates with acceleration component 116 via a link 118 (e.g.,network infrastructure 120).

Acceleration components in hardware acceleration plane 106 can be usedto accelerate larger-scale services robustly in a data center.Substantial portions of complex datacenter services can be mapped toacceleration components (e.g., FPGAs) by using low latency interconnectsfor computations spanning multiple acceleration components. Accelerationcomponents can also be reconfigured as appropriate to provide differentservice functionality at different times.

FIG. 2 illustrates an example architecture 202 that can be used in adata center. Servers 204, 206, and 208 can be included in a rack in thedata center. Each of servers 204, 206, and 208 can be coupled totop-of-rack (TOR) switch 210. Other racks, although not shown, may havea similar configuration. Server 204 further includes host component 212including CPUs 214, 216, etc. Host component 212 along with hostcomponents from servers 206 and 208 can be included in software plane104. Server 204 also includes acceleration component 218. Accelerationcomponent 218 along with acceleration components from servers 206 and208 can be included in hardware acceleration plane 106.

Acceleration component 218 is directly coupled to host component 212 vialocal link 220 (e.g., a PCIe link). Thus, host component 212 can viewacceleration component 218 as a local acceleration component andacceleration component 218 can view host component 212 as a local hostcomponent. Acceleration component 218 and host component 212 are alsoindirectly coupled by way of network interface controller 222 (e.g.,used to communicate across network infrastructure 120). Server 204 canload images representing service functionality onto accelerationcomponent 218.

Acceleration component 218 is also coupled to TOR switch 210. Hence, inarchitecture 202, acceleration component 218 represents the path throughwhich host component 212 interacts with other components in the datacenter (including other host components and other accelerationcomponents). Architecture 202 allows acceleration component 218 toperform processing on packets that are received from (and/or sent to)TOR switch 210 (e.g., by performing encryption, compression, etc.),without burdening the CPU-based operations performed by host component212.

Management functionality 232 serves to manage the operations ofarchitecture 202. Management functionality 232 can be physicallyimplemented using different control architectures. For example, in onecontrol architecture, the management functionality 232 may includeplural local management components that are coupled to one or moreglobal management components.

FIG. 3 illustrates an example server 302. Server 302 includes hostcomponent 304 including CPUs 306, 308, etc., acceleration component 310,and local link 312. Acceleration component 310 is directly coupled tohost component 304 via local link 312 (e.g., a PCIe link). Thus, hostcomponent 304 can view acceleration component 310 as a localacceleration component and acceleration component 310 can view hostcomponent 304 as a local host component. Host component 304 andacceleration component 310 can be included in software plane 104 andhardware acceleration plane 106 respectively. Server 302 implementsnetwork interface controller (NIC) 314 as an internal component ofacceleration component 310. Server 302 can load images representingservice functionality onto acceleration component 310.

FIG. 4 illustrates an example server 402. Server 402 includes hostcomponents 404 through 406 including any number n of host components.Host components 404 through 406 can be included in software plane 104.Server 402 includes acceleration components 408 through 410 includingany number m of acceleration components. Acceleration components 408through 410 can be included in hardware acceleration plane 106. Server402 can also include a network interface controller (not shown).

Server 402 can include a single host component locally linked to twoacceleration components. The two acceleration components can performdifferent respective tasks. For example, one acceleration component canbe used to process outgoing traffic to its local TOR switch, while theother acceleration component can be used to process incoming trafficfrom the TOR switch. In addition, server 402 can load imagesrepresenting service functionality onto any of the accelerationcomponents 408 through 410.

In general, a service (e.g., search ranking, encryption, compression,computer vision, speech translation, etc.) can be implemented at one ormore host components, at one or more acceleration components, or acombination of one or more host components and one or more accelerationcomponents depending on what components are better suited to providedifferent portions of the service.

FIG. 5 illustrates an example service 512 implemented using componentsof software plane 104 and components of hardware acceleration plane 106.In operation (1), host component 502 communicates with host component504 in the course of performing a computational task. In operation (2)host component 504 then requests the use of service 512 that isimplemented in the hardware acceleration plane 106 (although hostcomponent 504 may not be “aware” of where service 512 is implemented) bycommunicating with acceleration component 506 over a local link.

The requested service 512 is a composed service spread out over aplurality of acceleration components, each of which performs a specifiedportion of the service. Although acceleration component 506 wascontacted to request use of the service 512, acceleration component 506may not be the head of the composed service (or even be part of themulti-component service). Instead, acceleration component 508 may be thehead component for the composed service.

As such, in operation (3), host component 504 indirectly communicateswith acceleration component 508 via acceleration component 506.Acceleration component 508 then performs its portion of the composedservice to generate an intermediate output result. In operation (4),acceleration component 508 then invokes acceleration component 510,which performs another respective portion of the composed service, togenerate a final result. In operations (5), (6), and (7), the hardwareacceleration plane 106 successively forwards the final result back tothe requesting host component 504, through the same chain of componentsset forth above but in the opposite direction.

Operations in hardware acceleration plane 106 are performed in anindependent manner of operations performed in the software plane 104. Inother words, the host components in the software plane 104 do not managethe operations in the hardware acceleration plane 106. However, the hostcomponents may invoke the operations in the hardware acceleration plane106 by issuing requests for services that are hosted by the hardwareacceleration plane 106.

The hardware acceleration plane 106 operates in a manner that istransparent to a requesting host component. For example, host component504 may be “unaware” of how its request is being processed in hardwareacceleration plane 106, including the fact that the service correspondsto a composed service.

Communication in software plane 104 (e.g., corresponding to operation(1)) can take place using the same common network infrastructure 120 ascommunication in the hardware acceleration plane 106 (e.g., correspondto operations (3)-(6)). Operations (2) and (7) may take place over alocal link, corresponding to the local_(H)-to-local_(S) coupling 114shown in FIG. 1.

FIG. 6 illustrates an example service 612 implemented using componentsof a software plane and components of hardware acceleration plane 106Service 612 uses a different flow structure than service 512. Morespecifically, in operation (1), a host component (not shown) sends arequest to its local acceleration component 602. In this example, localacceleration component 602 is also the head component of service 612. Inoperation (2), local acceleration component 602 may then forward one ormore messages to a plurality of respective acceleration components. Eachacceleration component that receives a message may perform a portion ofa composed service in parallel with the other acceleration components.(FIG. 6 may represent only a portion of service 612, other portions ofservice 612 can be implemented at other hardware accelerators).

In general, an acceleration component can include any of variety ofcomponents some of which can be more or less common across differentapplication images. Some components, such as, for example, a role, aredistinct between application images. Other components, such as, forexample, routers, transport components, switches, diagnostic recorders,etc., can be relatively common between some number of applicationimages. These other relatively common components can be viewed as beingincluded in an intermediate layer of abstraction or “soft shell”.Further components, such as, for example, bridges, bypass controls,Network Interface Cards, Top of Rack Interfaces, buffers, memorycontrollers, PCIe controllers, Inter-FPGA network controllers,configuration memories and interfaces, host interfaces, debugging andback-channel interfaces (e.g., Joint Test Action Group (JTAG)interfaces, Inter-Integrated Circuit (I2C) interfaces, etc.), sensors,etc. can be very common between a higher number of (and essentially all)application images. These further very common components can be viewedas included in a greater layer of abstraction (e.g., than the otherrelatively common components) or “shell”.

When an FPGA is reconfigured with new functionality, it is likely(although not guaranteed) that a role (i.e., the application-specificlogic) at the FGPA is changed. However, it is unlikely that existingfunctionality in the soft shell is changed and it is extremely unlikelythat existing functionality in the soft shell is changed. Thus,components in the soft shell and to greater extent components in theshell provide a common interface for a role. As such, the shell allowscode for a role to be ported relatively easy across differentacceleration components.

Turning to FIG. 7, FIG. 7 illustrates an example architecture of anacceleration component 702. Acceleration component 702 can be includedin hardware acceleration plane 106. Components included in accelerationcomponent 702 can be implemented on hardware resources (e.g., logicblocks and programmable interconnects) of acceleration component 702.

Acceleration component 702 includes application logic 706, soft shell704 associated with a first set of resources and shell 711 associatedwith a second set of resources. The resources associated with shell 711correspond to lower-level interface-related components that generallyremain the same across many different application scenarios. Theresources associated with soft shell 704 can remain the same across atleast some different application scenarios. Application logic 706 may befurther conceptualized as including an application domain (e.g., a“role”). The application domain or role can represent a portion offunctionality included in a composed service spread out over a pluralityof acceleration components.

The application domain hosts application logic 706 that performs servicespecific tasks (such as a portion of functionality for rankingdocuments, encrypting data, compressing data, facilitating computervision, facilitating speech translation, machine learning, etc.).Resources associated with soft shell 704 are generally less subject tochange compared to the application resources, and the resourcesassociated with shell 711 are less subject to change compared to theresources associated with soft shell 704 (although it is possible tochange (reconfigure) any component of acceleration component 702).

In operation, application logic 706 interacts with the shell resourcesand soft shell resources in a manner analogous to the way asoftware-implemented application interacts with its underlying operatingsystem resources. From an application development standpoint, the use ofcommon shell resources and soft shell resources frees a developer fromhaving to recreate these common components for each service.

Referring first to shell 711, shell resources include bridge 708 forcoupling acceleration component 702 to the network interface controller(via a NIC interface 710) and a local top-of-rack switch (via a TORinterface 712). Bridge 708 also includes a data path that allows trafficfrom the NIC or TOR to flow into acceleration component 702, and trafficfrom the acceleration component 702 to flow out to the NIC or TOR.Internally, bridge 708 may be composed of various FIFOs (714, 716) whichbuffer received packets, and various selectors and arbitration logicwhich route packets to their desired destinations. A bypass controlcomponent 718, when activated, can control bridge 708 so that packetsare transmitted between the NIC and TOR without further processing bythe acceleration component 702.

Memory controller 720 governs interaction between the accelerationcomponent 702 and local memory 722 (such as DRAM memory). The memorycontroller 720 may perform error correction as part of its services.

Host interface 724 provides functionality that enables accelerationcomponent 702 to interact with a local host component (not shown). Inone implementation, the host interface 724 may use Peripheral ComponentInterconnect Express (PCIe), in conjunction with direct memory access(DMA), to exchange information with the local host component. The outershell may also include various other features 726, such as clock signalgenerators, status LEDs, error correction functionality, and so on.

Turning to soft shell 704, router 728 is for routing messages betweenvarious internal components of the acceleration component 702, andbetween the acceleration component and external entities (e.g., via atransport component 730). Each such endpoint is associated with arespective port. For example, router 728 is coupled to memory controller720, host interface 724, application logic 706, and transport component730.

Transport component 730 formulates packets for transmission to remoteentities (such as other acceleration components), and receives packetsfrom the remote entities (such as other acceleration components). A3-port switch 732, when activated, takes over the function of the bridge708 by routing packets between the NIC and TOR, and between the NIC orTOR and a local port associated with the acceleration component 702.

Diagnostic recorder 734 can store information regarding operationsperformed by the router 728, transport component 730, and 3-port switch732 in a circular buffer. For example, the information may include dataabout a packet's origin and destination IP addresses, host-specificdata, timestamps, etc. A technician may study a log of the informationin an attempt to diagnose causes of failure or sub-optimal performancein the acceleration component 702.

A plurality of acceleration components similar to acceleration component702 can be included in hardware acceleration plane 106.

Acceleration components can use different network topologies (instead ofusing common network infrastructure 120 for communication) tocommunicate with one another. In one aspect, acceleration components areconnected directly to one another, such as, for example, in a twodimensional torus.

FIG. 8 illustrates an acceleration component 802 including separateconfigurable domains 804, 806, etc. A configuration component (notshown) can configure each configurable domain without affecting otherconfigurable domains. Hence, the configuration component can configureone or more configurable domains while the other configurable domainsare executing operations based on their respective configurations, whichare not disturbed.

FIG. 9 illustrates functionality for performing data transfer between ahost component 902 and an associated (e.g., locally linked) accelerationcomponent 904. Data can be transferred via a host interface (e.g., hostinterface 724), for example, using PCIe in conjunction with DMA memorytransfer). In operation (1), host logic 906 places data to be processedinto kernel-pinned input buffer 908 in main memory associated with thehost logic 906. In operation (2), the host logic 906 instructs theacceleration component 904 to retrieve the data and begin processing it.The host logic's thread is then either put to sleep until it receives anotification event from the acceleration component 904, or it continuesprocessing other data asynchronously. In operation (3), the accelerationcomponent 904 transfers the data from the host logic's memory and placesit in an acceleration component input buffer 910.

In operations (4) and (5), the application logic 912 retrieves the datafrom the input buffer 910, processes it to generate an output result,and places the output result in an output buffer 914. In operation (6),the acceleration component 904 copies the contents of the output buffer914 into output buffer 916 (in the host logic's memory). In operation(7), acceleration component 904 notifies the host logic 906 that thedata is ready for it to retrieve. In operation (8), the host logicthread wakes up and consumes the data in the output buffer 916. Hostlogic 906 may then discard the contents of the output buffer 916, whichallows the acceleration component 904 to reuse it in the next loadingoperation.

FIG. 10 illustrates an example architecture of a host component 1002.Host component 1002 can include one or more processing devices 1004,such as one or more central processing units (CPUs). Host component 1002can also include any storage resources 1006 for storing any kind ofinformation, such as code, settings, data, etc. Without limitation, forinstance, storage resources 1006 may include any of RAM of any type(s),ROM of any type(s), flash devices, hard disks, optical disks, and so on.More generally, any storage resource can use any technology for storinginformation. Further, any storage resource may provide volatile ornon-volatile retention of information. Further, any storage resource mayrepresent a fixed or removable component of host component 1002. In onecase, host component 1002 may perform any of the operations associatedwith local tenant functionality when processing devices 1004 carry outassociated instructions stored in any storage resource or combination ofstorage resources. Host component 1002 also includes one or more drivemechanisms 1008 for interacting with any storage resource, such as ahard disk drive mechanism, an optical disk drive mechanism, and so on.

Host component 1002 also includes an input/output module 1010 forreceiving various inputs (via input devices 1012), and for providingvarious outputs (via output devices 1014). One particular outputmechanism may include a presentation device 1016 and an associatedgraphical user interface (GUI) 1018. Host component 1002 can alsoinclude one or more network interfaces 1020 for exchanging data withother devices via one or more communication conduits 1022. One or morecommunication buses 1024 communicatively couple the above-describedcomponents together.

Communication conduit(s) 1022 can be implemented in any manner, e.g., bya local area network, a wide area network (e.g., the Internet),point-to-point connections, etc., or any combination thereof.Communication conduit(s) 1022 can include any combination of hardwiredlinks, wireless links, routers, gateway functionality, name servers,etc., governed by any protocol or combination of protocols.

A plurality of host components similar to host component 1002 can beincluded in software plane 104.

FIG. 11 illustrates an example architecture 1102 of managementfunctionality 1122 for managing services in a data center. Architecture1102 can be included in architecture 102. As such, managementfunctionality 1122 as well as other associated components can beimplemented on hardware resources of a host component (e.g., in softwareplane 104) and/or implemented on hardware resources of an accelerationcomponent (e.g., in hardware acceleration plane 106). Host componenthardware resources can include any of the hardware resources associatedwith host component 1002. Acceleration component hardware resources caninclude any of the hardware resources associated with accelerationcomponent 702.

Management functionality 1122 can include a number of sub-componentsthat perform different respective functions (which can be physicallyimplemented in different ways). A local determination component 1124,for example, can identify the current locations of services withinarchitecture 102, based on information stored in a data store 1126. Inoperation, location determination component 1124 may receive a requestfor a service. In response, location determination component 1124returns an address associated with the service, if that address ispresent in data store 1126. The address may identify a particularacceleration component in hardware acceleration plane 106 that hosts (oris the head of) the requested service.

Request handling component (RHC) 1128 processes requests for servicesmade by instances of tenant functionality. For example, an instance oftenant functionality may correspond to a software program running on aparticular local host component. That software program may request aservice in the course of its execution. The RHC 1128 handles the requestby determining an appropriate component in architecture 102 to providethe service. Possible components for consideration include: a localacceleration component (associated with the local host component fromwhich the request originated); a remote acceleration component; and/orthe local host component itself (whereupon the local host componentimplements the service in software). RHC 1128 makes its determinationsbased on one or more request handling considerations, such as whetherthe requested service pertains to a line-rate service. Further, the RHC1128 may interact with the location determination component 1124 inperforming its functions.

A global service allocation component (GSAC) 1130 can operate in abackground and global mode, allocating services to accelerationcomponents based on global conditions in architecture 102 (rather thanhandling individual requests from instances of tenant functionality, asdoes RHC 1128). For example, GSAC 1130 may invoke its allocationfunction in response to a change in demand that affects one or moreservices. GSAC 1130 makes its determinations based on one or moreallocation considerations, such as the historical demand associated withthe services, etc. Further, the GSAC 1130 may interact with the locationdetermination component 1124 in performing its functions. Asub-component of GSAC 1130 can also manage multi-component and/orcomposed services. A multi-component service is a service that iscomposed of plural parts. Plural respective acceleration componentsperform the respective parts.

FIG. 12 illustrates an example architecture with additional componentsof management functionality 1122. As described, location determinationcomponent 1124 identifies the current location of services withinarchitecture 102, based on information stored in the data store 1126. Inoperation, the location determination component 1124 receives a requestfor a service. In response, it returns the address of the service, ifpresent within the data store 1126. The address may identify aparticular acceleration component that implements the service.

Request handling component (RHC) 1128 handles requests for services bytenant functionality that resides on the host components. In response toeach request by a local host component, RHC 1128 determines anappropriate component to implement the service. For example, RHC 1128may choose from among: a local acceleration component (associated withthe local host component that made the request), a remote accelerationcomponent, or the local host component itself (whereupon the local hostcomponent will implement the service in software), or some combinationthereof. RHC 1128 performs its determinations based on one or morerequest handling considerations.

General allocation component (GSAC) 1130, on the other hand, operates byglobally allocating services to acceleration components withinarchitecture 102 to meet overall anticipated demand in the dataprocessing system and/or to satisfy other objectives (rather thanindividual requests by host components). In performing its functions,the GSAC component 1130 may draw on a data store 1202 that provides freepool information. The free pool information identifies accelerationcomponents that have free capacity to implement one or more services.The GSAC 1130 can also receive input information that has a bearing onits allocation decisions. One such piece of input information pertainsto historical demand information associated with a service, e.g., asmaintained in a data store 1204.

GSAC 1130 and RHC 1128 may use, in part, common logic in reaching theirallocation decisions, and that common logic may, in part, taken intoaccount similar allocation considerations. Further, both RHC 1128 andGSAC 1130 interact with the location determination component 124 in thecourse of performing their respective operations. Otherwise, asdescribed, the GSAC 1130 frames its decisions in a global context,whereas the RHC 1128 is an on-demand component that is primarily focusedon satisfying specific requests.

Configuration component 1206 configures acceleration components, e.g.,by sending a configuration steam to the acceleration components. Aconfiguration stream specifies the logic (e.g., an image) to be“programmed” into a recipient acceleration component. The configurationcomponent 1206 may use different strategies to configure an accelerationcomponent.

The failure monitoring component 1208 determines whether a previouslyconfigured acceleration component has failed. Various components of themanagement functionality 1122 may respond to failure notification bysubstituting a spare acceleration component for a failed accelerationcomponent, reconfiguring an acceleration component, partialreconfiguring acceleration component, reloading data in an accelerationcomponent, etc.

As described, functionality for a service or portion thereof can beprovided by linking roles from a group of interoperating accelerationcomponents. The linked roles can be composed in a directed graph in anyvariety of different ways, including a directed acyclic graph, adirected cyclic graph, etc., to provide service functionality and/oracceleration. For example, in some aspects, linked roles are composed ina pipeline or ring.

FIGS. 13A and 13B illustrate an example architecture 1300 for allocatingacceleration component functionality to support a service. Referringinitially to FIG. 13A, computer architecture 1300 includes servicemanager 1321. Service manager 1321 can be connected to (or be part of)network infrastructure 120, such as, for example, a Local Area Network(“LAN”), a Wide Area Network (“WAN”), and even the Internet.Accordingly, service manager 1321, host components in software plane104, acceleration components in hardware acceleration plane 106 and anyother connected computer systems and their components, can createmessage related data and exchange message related data (e.g., InternetProtocol (“IP”) datagrams and other higher layer protocols that utilizeIP datagrams, such as, Transmission Control Protocol (“TCP”), HypertextTransfer Protocol (“HTTP”), Simple Mail Transfer Protocol (“SMTP”),Simple Object Access Protocol (SOAP), etc. or using other non-datagramprotocols) over network infrastructure 120.

In general, configuration service 1321 is configured to monitor networkinfrastructure 120 and allocate groups of interoperating accelerationcomponents in hardware acceleration plane 106 to accelerate services.Configuration service 1321 can include any of the functionalitydescribed with respect to management functionality 232 and managementfunctionality 1122.

Turning to FIG. 14, FIG. 14 illustrates a flow chart of an examplemethod 1400 for allocating acceleration component functionality tosupport a service. Method 1400 will be described with respect to thecomponents and data of architecture 1300.

Method 1400 includes monitoring characteristics of a plurality ofhardware acceleration components in a hardware acceleration plane, thehardware acceleration plane providing a configurable fabric ofacceleration components for accelerating services (1401). For example,service manager 1321 can monitor the characteristics of accelerationcomponents in hardware acceleration plane 106. Within hardwareacceleration plane 106, service manager 1321 can monitor one or more of:load, utilization, failures, machine liveness, and service demand.

Method 1400 includes allocating a group of interoperating accelerationcomponents, from among the plurality of hardware accelerationcomponents, to provide service acceleration for a service, the group ofinteroperating acceleration components allocated based on the monitoredcharacteristics in view of an allocation policy for the hardwareacceleration plane, a role at each acceleration component in the groupof interoperating acceleration components linked together to compose agraph providing the service acceleration (1402). For example, servicemanager 1321 (e.g., using global service allocation component (GSAC)1130 or another similar component) can allocate acceleration components1301, 1302, 1303, and 1304 to provide acceleration for a service basedon monitored characteristics of hardware acceleration plane 104 and inview of allocation policy 1326. Allocation policy 1326 can be defined tofacilitate one or more of: balancing load across acceleration componentsin hardware acceleration plane 106, minimizing switching of roles atacceleration components in hardware acceleration plane 106, addressingchanges for demand in service acceleration, minimizing communicationlatency between acceleration components in hardware acceleration plane106 (e.g., by selecting physical adjacent acceleration components),grouping based on role, grouping based on network bandwidth requirementsfor a service, grouping for power efficiency, etc. A role at each ofacceleration components 1301, 1302, 1303, and 1304 can be linkedtogether to compose graph 1322.

Method 1400 includes maintaining an address for the graph so that theservice can request hardware acceleration from the group ofinteroperating acceleration components (1403). For example, servicemanager 1321 can maintain address 1327 for acceleration component 1301(the head of graph 1322) in storage 1351. Thus, when service manager1321 receives a subsequent request for service acceleration that can besatisfied by graph 1322, service manager 1321 can use address 1327 todirect the request to graph 1322.

Turning to FIG. 13B, FIG. 13B depicts a more detailed view of graph1322. As depicted, graph 1322 includes acceleration components 1301,1302, 1303, and 1304. Acceleration components 1301, 1302, 1303, and 1304can be FPGAs. The depicted arrangement of acceleration components 1301,1302, 1303, and 1304 is logical. The physical proximity of accelerationcomponents 1301, 1302, 1303, and 1304 relative to one another can vary(e.g., same server, different servers same rack, different racks, etc.).

Acceleration components 1301, 1302, 1303, and 1304 are programmed withcorresponding roles 1311, 1312, 1313, and 1314 respectively. Roles 1311,1312, 1313, and 1314 are linked to one another to compose graph 1322.Graph 1322 can provide hardware acceleration for a service.

Input and output from roles at (logically) neighboring accelerationcomponents may depend on one another or input and output from othercomponents (e.g., host components or functionality composed from adifferent group of interoperating acceleration components). For example,input to role 1312 can depend on output from role 1311. Similarly, inputto role 1313 can depend on output from role 1312.

Some or all of acceleration components 1301, 1302, 1303, and 1304 canparticipate in one-way or two-way communication with (logically)neighboring acceleration components and other components (e.g., hostcomponents). Thus, input and output from one acceleration component candepend on input and output from another acceleration component and viceversa. For example, input to role 1311 can depend on output from role1312 and input to role 1312 can depend on output from role 1313.

A graph can be composed from a linked roles at a group of interoperatingacceleration components. For example, graph 1322 can provideacceleration for part of a document ranking service used to providesearch engine results. Graph 1322 can interoperate with other portionsof service functionality composed from other groups of interoperatingacceleration components and/or provided by one or more host components.For example, for a document ranking service, document feature extractioncan be composed from one group of interoperating accelerationcomponents, free form expression calculations can be composed fromanother group of interoperating acceleration components, and scoringcalculations can be composed from a further group of interoperatingacceleration components.

Thus, other graphs in hardware acceleration plane 106 can provideacceleration for another part of the document ranking service or canprovide acceleration for some other service (e.g., encryption,compression, computer vision, speech translation, etc.).

Service manager 1321 can monitor the performance of graph 1322. Servicemanager 1321 can consider the performance of graph 1322 when allocatinggroups of interoperating acceleration components for accelerating otherparts of a service and for accelerating other services. For example, ifservice manager 1321 detects an error in graph 1322, service manager mayneed to allocate a different group of interoperating accelerationcomponents to provide graph 1322.

In response, service manage 1321 can send a halt command to graph 1322.The halt command halts operation of one or more of roles 1311, 1312,1313, and 1314 as well as quiescing graph 1322. Quiescing graph 1322stops all network traffic flowing into and out of graph 1322. A haltcommand can also instructs roles within graph 1322 to ignore any datacoming out of other roles in graph 1322.

When appropriate, service manager 1321 can also notify upstream anddownstream components of roles (e.g., in the same graph) that anotherrole is being halted. Upstream and downstream components can then takeappropriate actions with in-process data (e.g., buffer, drop, send NACKsto other components, etc.) until the role is again operational.

Turning to FIG. 13C, service 1341 can use address 1327 to determine thatacceleration component 1301 is the head of graph 1322. Service 1341 cansend acceleration request 1342 to acceleration component 1301.Acceleration request 1342 can be a request to process data 1343 (e.g., aset of documents for document ranking). Data 1343 can be processed byroles 1311, 1312, 1313, and 1314 (and possibly passed off to andreturned from other other acceleration components) to formulate results1344 (a set of ranked documents). Results 1344 can be returned back toservice 1341.

In general, an acceleration component (e.g., 1301, 1302, 1303, 1304,etc.) can include an array of programmable logic blocks and a hierarchyof reconfigurable interconnects that allow logic blocks to be connectedtogether in different configurations (to provide different functionality(i.e., different roles)). Image files can be loaded at an accelerationcomponent to configure programmable logic blocks and configureinterconnects to provide desired functionality. Images can be stored ata network location and/or local to an acceleration component or locallylinked host component.

FIGS. 15A-15D illustrate an example architecture for allocatingacceleration component functionality to support services. Referringinitially to FIG. 15A, computer architecture 1500 includes servicemanager 1521. Service manager 1521 can be connected to (or be part of)network infrastructure 120, such as, for example, a Local Area Network(“LAN”), a Wide Area Network (“WAN”), and even the Internet.Accordingly, service manager 1521, host components in software plane104, acceleration components in hardware acceleration plane 106 and anyother connected computer systems and their components, can createmessage related data and exchange message related data (e.g., InternetProtocol (“IP”) datagrams and other higher layer protocols that utilizeIP datagrams, such as, Transmission Control Protocol (“TCP”), HypertextTransfer Protocol (“HTTP”), Simple Mail Transfer Protocol (“SMTP”),Simple Object Access Protocol (SOAP), etc. or using other non-datagramprotocols) over network infrastructure 120.

Hardware acceleration plane 106 includes a finite number of accelerationcomponents. Service manager 1521 can allocate the finite number ofacceleration components to compose various different graphs foraccelerating services. Service manager 1521 can monitor characteristicsof network infrastructure 120. Service manager 1521 can allocate groupsof interoperating acceleration components to compose graphs based on themonitored characteristics and in view of allocation policy 1526.

In general, configuration service 1521 is configured to monitor networkinfrastructure 120 and allocate groups of interoperating accelerationcomponents in hardware acceleration plane 106 to accelerate services.Configuration service 1521 can include any of the functionalitydescribed with respect to management functionality 232 and managementfunctionality 1122.

As depicted in FIG. 15A, acceleration components in hardwareacceleration plane 106 are allocated to compose a copy of graph 1501, acopy of graph 1502, two copies of graph 1503, and a copy of graph 1504.Addresses 1527, 1528, etc. stored in storage 1541 can be used to directrequests for acceleration to the appropriate graphs.

During continued monitoring, service manager 1521 can detect anincreased demand for graph 1502. Based on the current allocation ofacceleration components in hardware plane 106 and in view of allocationpolicy 1526, service manager 1521 can (re)allocate accelerationcomponents to meet the increased demand for graph 1502. Service manager1521 can (re)allocate acceleration components in a manner that compliesto an extent possible with allocation policy 1526 (e.g., balances loadin hardware acceleration plane 106, minimize role switching, etc.).

Turning to FIG. 15B, in one aspect, service manager 1521 allocatesunused (spare) acceleration components to compose another copy of graph1502. Addresses in storage 1541 can be adjusted so that the other copyof graph 1502 can be accessed.

Turning to FIG. 15C, in another aspect, service manager 1521reconfigures (e.g., loads images files at) a group of interoperatingacceleration components to compose together another copy of graph 1502.Reconfiguration removes graph 1501. There may be so little demand forgraph 1501 that requests for graph 1501 can be temporarily queued upuntil another copy of graph 1501 can be allocated elsewhere in hardwareplane 106. Addresses in storage 1541 can be adjusted so that the othercopy of graph 1502 can be accessed and so that reference to graph 1501is removed.

Turning to FIG. 15D, in another aspect, service manager 1521 changes agroup of interoperating acceleration components composed to providegraph 1503 into a group of interoperating acceleration componentscomposed to provide graph 1502. The group of interoperating accelerationcomponents can be configured with roles for providing both graph 1502and graph 1503. Service manager 1521 can send configuration data tochange the roles from providing graph 1503 to providing graph 1502.Configuration data can used to switch between roles (or otherwise changebehavior) at an acceleration component using the same underlying image.As such, roles (or other behavior) can be changed without having to loada new image file. Addresses in storage 1541 can be adjusted so that theother copy of graph 1502 can be accessed and so that just the remainingcopy of graph of 1503 can be accessed.

The graphs in FIGS. 15A-15D can be for accelerating completely differentservices, such as, for example, document ranking, data encryption, datacompression, speech translation, and computer vision. In another aspect,some of the graphs in FIGS. 15A-15D can be for accelerating differenttypes of similar or even the same service. For example, graph 1502 canbe for accelerating document ranking for documents in French and graph1503 can be for accelerating document ranking for documents in English.Graphs 1501 and 1504 can be for accelerating other different services.

In one aspect, service manager 1521 allocates a plurality of graphs ofthe same type (kind) and then balances load between the plurality ofgraphs. For example, service manager 1521 can allocate two graphs 1502for accelerating ranking documents in French. Service manager 1521(e.g., using request handling component (RHC) 1128 or another similarcomponent) can then route requests for ranking documents in Frenchbetween the two graphs 1502 based on any the described monitoredcharacteristics and in accordance with any of the described allocationpolicies.

In general, the described aspects are advantageous for using a finitenumber of acceleration components to accelerate services. Accelerationcomponents can be allocated in a manner that balances load in a hardwareacceleration plane, minimizing role switching, and adapts to demandchanges. When role switching is appropriate, lower overhead mechanisms(e.g., configuration data) can be used to switch roles to the extentpossible.

In some aspects, a system includes a hardware acceleration plane, asoftware plane, and a network infrastructure. The hardware accelerationplane includes a configurable fabric of a plurality of accelerationcomponents (e.g., hardware accelerators, such as, FPGAs). The softwareplane includes a plurality of host components (e.g., CPUs) runningsoftware. The network infrastructure is shared by accelerationcomponents in the hardware acceleration plane and host components in thesoftware plane. The network infrastructure is used by accelerationcomponents to communicate directly with one another. Local links connectacceleration components and host components (e.g., in the same server).

The system also includes one or more computer storage devices havingstored thereon computer-executable instructions representing a servicemanager. The service manager is configured to allocate accelerationcomponent functionality for supporting a service. Allocatingacceleration component functionality includes monitoring characteristicsof the plurality of hardware acceleration components.

Allocating acceleration component functionality includes allocating agroup of interoperating acceleration components, from among theplurality of hardware acceleration components, to provide serviceacceleration for a service. The group of interoperating accelerationcomponents is allocated based on the monitored characteristics in viewof an allocation policy for the hardware acceleration plane. A role ateach acceleration component in the group of interoperating accelerationcomponents linked together to compose a graph providing the serviceacceleration. Allocating acceleration component functionality includesmaintaining an address for the graph so that the service can requesthardware acceleration from the group of interoperating accelerationcomponents.

In another aspect, a method for allocating acceleration componentfunctionality for supporting a service. Characteristics of the pluralityof hardware acceleration components are monitored. The hardwareacceleration plane provides a configurable fabric of accelerationcomponents for accelerating services.

A group of interoperating acceleration components, from among theplurality of hardware acceleration components, is allocated to provideservice acceleration for a service. The group of interoperatingacceleration components is allocated based on the monitoredcharacteristics in view of an allocation policy for the hardwareacceleration plane. A role at each acceleration component in the groupof interoperating acceleration components linked together to compose agraph providing the service acceleration. An address for the graph ismaintained so that the service can request hardware acceleration fromthe group of interoperating acceleration components.

In another aspect, a computer program product for use at a computersystem includes one or more computer storage devices having storedthereon computer-executable instructions that, in response to executionat a processor, cause the computer system to implement a method forallocating acceleration component functionality for supporting aservice.

The computer program product includes computer-executable instructionsthat, in response to execution at a processor, monitor characteristicsof a plurality of hardware acceleration components in a hardwareacceleration plane. The hardware acceleration plane provides aconfigurable fabric of acceleration components for acceleratingservices.

The computer program product includes computer-executable instructionsthat, in response to execution at a processor, allocate a group ofinteroperating acceleration components, from among the plurality ofhardware acceleration components, to provide service acceleration for aservice. The group of interoperating acceleration components isallocated based on the monitored characteristics in view of anallocation policy for the hardware acceleration plane. A role at eachacceleration component in the group of interoperating accelerationcomponents is linked together to compose a graph providing the serviceacceleration. The computer program product includes computer-executableinstructions that, in response to execution at processor, maintain anaddress for the graph so that the service can request hardwareacceleration from the group of interoperating acceleration components.

The present described aspects may be implemented in other specific formswithout departing from its spirit or essential characteristics. Thedescribed aspects are to be considered in all respects only asillustrative and not restrictive. The scope is, therefore, indicated bythe appended claims rather than by the foregoing description. Allchanges which come within the meaning and range of equivalency of theclaims are to be embraced within their scope.

What is claimed:
 1. A system comprising: a pool of accelerationcomponents configurable to accelerate a service comprising at least afirst service functionality and a second service functionality,different from the first functionality; a first graph comprising a firstset of interconnected acceleration components, wherein the first graphis configured to accelerate the first service functionalitycorresponding to the service, and wherein each of the first set ofacceleration components comprises a programmable logic array comprisinghardware logic blocks interconnected using reconfigurable interconnects;a second graph comprising a second set of interconnected accelerationcomponents, wherein the second graph is configured to accelerate thesecond service functionality corresponding to the service, and whereineach of the second set of acceleration components comprises aprogrammable logic array comprising hardware logic blocks interconnectedusing reconfigurable interconnects; and a service manager comprisingcomputer-executable instructions configured to: (1) add, based on anallocation policy, in response to a first increased demand for hardwareacceleration from the service determined by monitoring a firstcharacteristic corresponding to the first set of interconnectedacceleration components, at least one more acceleration component to thefirst set of interconnected acceleration components from the pool ofacceleration components, (2) add, based on the allocation policy, inresponse to a second increased demand for hardware acceleration from theservice determined by monitoring a second characteristic, different fromthe first characteristic, corresponding to the second set ofinterconnected acceleration components, at least one more accelerationcomponent to the second set of interconnected acceleration componentsfrom the pool of acceleration components.
 2. The system of claim 1,wherein each of the first graph and the second graph provides serviceacceleration for a service selected from among: document ranking, dataencryption, data compression, speech translation, computer vision, ormachine learning.
 3. The system of claim 1, wherein the allocationpolicy relates to balancing load among acceleration components orminimizing role switching among acceleration components.
 4. The systemof claim 1, wherein each of the first set of interconnected accelerationcomponents comprises at least one field programmable gate array (FPGA).5. The system of claim 1, wherein each of the second set ofinterconnected acceleration components comprises at least one fieldprogrammable gate array (FPGA).
 6. The system of claim 1, wherein eachof the first characteristic and the second characteristic is one ofload, utilization, failures, or machine liveness.
 7. The system of claim1, wherein the service manager further comprises computer-executableinstructions configured to detect an error associated with the firstgraph and the second graph.
 8. A method in a data center comprising apool of acceleration components for accelerating the service comprisingat least a first service functionality and a second servicefunctionality, different from the first functionality, the methodcomprising: forming a first graph comprising a first set ofinterconnected acceleration components, wherein the first graph isconfigured to accelerate the first service functionality correspondingto the service, and wherein each of the first set of accelerationcomponents comprises a programmable logic array comprising hardwarelogic blocks interconnected using reconfigurable interconnects; forminga second graph comprising a second set of interconnected accelerationcomponents, wherein the second graph is configured to accelerate thesecond service functionality corresponding to the service, and whereineach of the second set of acceleration components comprises aprogrammable logic array comprising hardware logic blocks interconnectedusing reconfigurable interconnects; in response to a first increaseddemand for hardware acceleration from the service determined bymonitoring a first characteristic corresponding to the first set ofinterconnected acceleration components, based on an allocation policy, aservice management component adding at least one more accelerationcomponent to the first set of interconnected acceleration componentsfrom the pool of acceleration components; and in response to a secondincreased demand for hardware acceleration from the service determinedby monitoring a second characteristic, different from the firstcharacteristic, corresponding to the second set of interconnectedacceleration components, based on the allocation policy, the servicemanagement component adding at least one more acceleration component tothe second set of interconnected acceleration components from the poolof acceleration components.
 9. The method of claim 8, wherein each ofthe first graph and the second graph provides service acceleration for aservice selected from among: document ranking, data encryption, datacompression, speech translation, computer vision, or machine learning.10. The method of claim 8, wherein the allocation policy relates tobalancing load among acceleration components or minimizing roleswitching among acceleration components.
 11. The method of claim 8,wherein each of the first set of interconnected acceleration componentscomprises at least one field programmable gate array (FPGA).
 12. Themethod of claim 8, wherein each of the second set of interconnectedacceleration components comprises at least one field programmable gatearray (FPGA).
 13. The method of claim 8, wherein each of the firstcharacteristic and the second characteristic is one of load,utilization, failures, or machine liveness.
 14. The method of claim 8further comprising detecting an error associated with the first graph orthe second graph.
 15. A non-transitory computer program productcomprising instructions configured to perform a method in a data centercomprising a pool of acceleration components for accelerating theservice comprising at least a first service functionality and a secondservice functionality, different from the first functionality, themethod comprising: forming a first graph comprising a first set ofinterconnected acceleration components, wherein the first graph isconfigured to accelerate the first service functionality correspondingto the service, and wherein each of the first set of accelerationcomponents comprises a programmable logic array comprising hardwarelogic blocks interconnected using reconfigurable interconnects; forminga second graph comprising a second set of interconnected accelerationcomponents, wherein the second graph is configured to accelerate thesecond service functionality corresponding to the service, and whereineach of the second set of acceleration components comprises aprogrammable logic array comprising hardware logic blocks interconnectedusing reconfigurable interconnects; in response to a first increaseddemand for hardware acceleration from the service determined bymonitoring a first characteristic corresponding to the first set ofinterconnected acceleration components, based on an allocation policy, aservice management component adding at least one more accelerationcomponent to the first set of interconnected acceleration componentsfrom the pool of acceleration components; and in response to a secondincreased demand for hardware acceleration from the service determinedby monitoring a second characteristic, different from the firstcharacteristic, corresponding to the second set of interconnectedacceleration components, based on the allocation policy, the servicemanagement component adding at least one more acceleration component tothe second set of interconnected acceleration components from the poolof acceleration components.
 16. The non-transitory computer programproduct of claim 15, wherein each of the first graph and the secondgraph provides service acceleration for a service selected from among:document ranking, data encryption, data compression, speech translation,computer vision, or machine learning.
 17. The non-transitory computerprogram product of claim 15, wherein the allocation policy relates tobalancing load among acceleration components or minimizing roleswitching among acceleration components.
 18. The non-transitory computerprogram product of claim 15, wherein each of the first set ofinterconnected acceleration components comprises at least one fieldprogrammable gate array (FPGA).
 19. The non-transitory computer programproduct of claim 15, wherein each of the second set of interconnectedacceleration components comprises at least one field programmable gatearray (FPGA).
 20. The non-transitory computer program product of claim15, wherein each of the first characteristic and the secondcharacteristic is one of load, utilization, failures, or machineliveness.